*: compatibility with staleread #24285

xhebox · 2021-04-26T06:54:13Z

What problem does this PR solve?

The main issue #22427

Problem Summary: Added a infoschema cache for stale read and history read. Now NewTxnWithStalenessOption will make use of the history infoschema from the cache.

Changes listed:

infoHandle to infoCache
infoHandle.IsValid() replaced by infoCache.Get() != nil
refactor on loadInfoSchema, tryLoadSchemaDiffs, Reload of domain
detach infoHandle from Builder, it now depends on kv.Storage
Remove some tests and cover by cache_test.go.
Remove the old staleSchemaVersion check, it is now expected to be compatible all the time
Fix tests by EXACT STALENESS '00:00:00'(need to read the newest infoschema updated by test preparation SQLs).

Note that #24230 and #24236 is needed for stale read, too. Otherwise, the provided old infoschema is not used in some cases.

Check List

Tests

Unit test
Integration test

Release note

No release note

xhebox · 2021-04-26T08:57:31Z

/hold

xhebox · 2021-04-26T11:19:06Z

/unhold

domain/domain.go

infoschema/cache.go

ddl/options.go

domain/domain.go

djshow832

Rest LGTM

infoschema/cache.go

xhebox · 2021-05-07T02:51:34Z

@djshow832 There is a capacity for the cache. Considering TiDB can already OOM with current schema and GC-managed snapshot schema, it is bounded with a capacity. Then it comes the problem of which schema to drop if there is not enough space, and I choose to keep newers and drop olders, the least recent, in another word.

JmPotato · 2021-05-10T06:53:30Z

ddl/ddl.go

@@ -649,10 +649,10 @@ func (d *ddl) startCleanDeadTableLock() {
 			if !d.ownerManager.IsOwner() {
 				continue
 			}
-			if d.infoHandle == nil || !d.infoHandle.IsValid() {
+			if d.infoCache == nil || d.infoCache.GetLatest() == nil {


What about providing a function to do the check rather than comparing it with nil every time?

Sounds good, but infoCache can be nil for tests. And is may not be loaded, that is why there are checks.

I will see if it can be ensured to be non-nil. I don't really like to add a method for ddlCtx.

All checks are removed. infoCache is promised to be non-nil.

ddl/partition.go

domain/domain.go

JmPotato · 2021-05-10T06:58:05Z

domain/domain.go

@@ -658,7 +636,7 @@ func NewDomain(store kv.Storage, ddlLease time.Duration, statsLease time.Duratio
 		exit:                make(chan struct{}),
 		sysSessionPool:      newSessionPool(capacity, factory),
 		statsLease:          statsLease,
-		infoHandle:          infoschema.NewHandle(store),
+		infoCache:           infoschema.NewCache(16),


Why 16? If it's universal, using a const variable to define it may be better.

It is just a guessing value. I want to propose a new proposal to remove this value in the feature. Or maybe get a instance variable for it in the next PR.

JmPotato · 2021-05-10T07:01:24Z

infoschema/cache.go

+
+// Insert will **TRY** to insert the infoschema into the cache.
+// It only promised to cache the newest infoschema.
+func (h *InfoCache) Insert(is InfoSchema) {


Do we have any mechanism to clean up the InfoSchema that is too old in cache?

If there is newer, the older will be dropped. But there is no aggeresive mechanism.

djshow832 · 2021-05-10T07:19:30Z

/lgtm

Signed-off-by: xhe <[email protected]>

Co-authored-by: xiongjiwei <[email protected]>

Signed-off-by: xhe <[email protected]>

Yisaer

Rest LGTM

Yisaer · 2021-05-13T09:20:57Z

executor/stale_txn_test.go

+	schemaVer1 := tk.Se.GetSessionVars().GetInfoSchema().SchemaMetaVersion()
+	tk.MustExec("drop table if exists t")
+	schemaVer2 := tk.Se.GetSessionVars().GetInfoSchema().SchemaMetaVersion()
+	// confirm schema changed
+	c.Assert(schemaVer1, Less, schemaVer2)
+
+	tk.MustExec(`START TRANSACTION READ ONLY WITH TIMESTAMP BOUND EXACT STALENESS '00:00:01'`)
+	schemaVer3 := tk.Se.GetSessionVars().GetInfoSchema().SchemaMetaVersion()
+	// got an old infoSchema
+	c.Assert(schemaVer3, Less, schemaVer2)
+}


I think we should sleep(1s) after we create job, and assert schemaVer3 equals schemaVer1 for this case.

Signed-off-by: xhe <[email protected]>

Yisaer

/lgtm

Yisaer · 2021-05-17T07:00:51Z

infoschema/cache.go

+// Insert will **TRY** to insert the infoschema into the cache.
+// It only promised to cache the newest infoschema.
+// It returns 'true' if it is cached, 'false' otherwise.
+func (h *InfoCache) Insert(is InfoSchema) bool {


Is it necessary to record metrics for inserting?

I was thinking about that. But I gave up. There are already LoadSchemaCounter or so. It may not be as important as the hit rate. Maybe when the infoCache become more complex.

ti-chi-bot · 2021-05-17T07:01:01Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Yisaer
djshow832

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

xhebox · 2021-05-17T07:20:59Z

/merge

xhebox · 2021-05-17T07:37:06Z

/merge cancel

Signed-off-by: xhe <[email protected]>

xhebox · 2021-05-17T11:00:23Z

/run-unit-test

Signed-off-by: xhe <[email protected]>

xhebox · 2021-05-17T11:35:48Z

/merge

ti-chi-bot · 2021-05-17T11:35:51Z

This pull request has been accepted and is ready to merge.

Commit hash: 3fa0aed

ti-chi-bot · 2021-05-17T11:42:10Z

@xhebox: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

…ingcap#24052) * *: fix revoke statement for CURRENT_USER() and refine error message planner: support set tidb_allow_mpp to `2` or `ENFORCE` to enforce use mpp mode. (pingcap#24516) store/tikv: remove use of SchemaAmender option in store/tikv (pingcap#24408) *: the value of tikv-client.store-liveness-timeout should not less than 0 (pingcap#24244) store/tikv: remove use of EnableAsyncCommit option in store/tikv (pingcap#24462) txn: Add txn state's view (pingcap#22908) planner: ignore lock for temporary table of PointGet and BatchPointGet (pingcap#24540) store/tikv: remove use of ReplicaRead transaction option in store/tikv (pingcap#24409) store/driver: move error to single package (pingcap#24549) ddl: add check table compatibility for temporary table (pingcap#24501) store/tikv: remove use of IsStatenessReadOnly option in store/tikv (pingcap#24464) store/tikv: change backoff type for missed tiflash peer. (pingcap#24577) store/tikv: remove use of MatchStoreLabels transaction option in store/tikv (pingcap#24465) executor, meta: Allocate auto id for global temporary tables (pingcap#24506) store/tikv: remove use of SampleStep option in store/tikv (pingcap#24461) executor: add partition pruning tests for adding and dropping partition operations (pingcap#24573) ddl: forbid partition on temporary mode before put into queue (pingcap#24565) ddl: speedup test case TestIndexOnMultipleGeneratedColumn (pingcap#24487) execution: Fix issue 24439 Inconsistent error with MySQL for GRANT CREATE USER ON <specific db>.* (pingcap#24485) *: fix errcheck (pingcap#24463) test: make TestExtractStartTs stable (pingcap#24585) ddl: forbid recover/flashback temporary tables (pingcap#24518) executor: fix point_get result on clustered index when new-row-format disabled but new-collation enabled (pingcap#24544) executor: Improve the performance of appending not fixed columns (pingcap#20969) *: typo fix (pingcap#24564) planner/core: refresh stale regions in cache for batch cop response (pingcap#24457) binlog: DML on temporary tables do not write binlog (pingcap#24570) store/tikv: remove use of GuaranteeLinearizability option in store/tikv (pingcap#24605) store/tikv: remove use of CollectRuntimeStats option in store/tikv (pingcap#24604) store/tikv: move Backoffer into a single package (pingcap#24525) variables: init cte max recursive deeps in a new session (pingcap#24609) store/tikv: move transaction options out to /kv (pingcap#24619) store/driver: move backoff driver into single package so we can use i… (pingcap#24624) server: close the temporary session in HTTP API to avoid memory leak (pingcap#24339) store/tikv: use latest PD TS plus one as min commit ts (pingcap#24579) planner: fix incorrect TableDual plan built from nulleq (pingcap#24596) ranger: fix the case which could have duplicate ranges (pingcap#24590) executor, store: Pass the SQL digest down to pessimistic lock request (pingcap#24380) kv: remove UnionStore interface (pingcap#24625) *: enable gosimple linter (pingcap#24617) txn: avoid the gc resolving pessimistic locks of ongoing transactions (pingcap#24601) util: fix wrong enum building for index range (pingcap#24632) sessionctx: change innodb large prefix default (pingcap#24555) store: fix data race about KVStore.tikvClient (pingcap#24655) executor, privileges: Add dynamic privileges to SHOW PRIVILEGES (pingcap#24646) ddl: refactor rule [4/6] (pingcap#24007) cmd: ddl_test modify retryCnt from 5 to 20 (pingcap#24662) executor: add correctness tests about direct reading with ORDER BY and LIMIT (pingcap#24455) store/tikv: remove options from unionstore (pingcap#24629) planner: fix wrongly check for update statement (pingcap#24614) store/tikv: remove CompareTS (pingcap#24657) planner, privilege: Add security enhanced mode part 4 (pingcap#24416) executor: add some test cases about partition table dynamic-mode with split-region (pingcap#24665) planner: fix wrong column offsets when processing dynamic pruning for IndexJoin (pingcap#24659) *: Add security enhanced mode part 3 (pingcap#24412) store/tikv: resolve ReplicaReadType dependencies (pingcap#24653) executor: add test cases about partition table with `expression` (pingcap#24628) tablecodec: fix write wrong prefix index value when collation is ascii_bin/latin1_bin (pingcap#24578) *: compatibility with staleread (pingcap#24285) session: test that temporary tables will also be retried (pingcap#24505) domain, session: Add new sysvarcache to replace global values cache (pingcap#24359) ddl, transaction: DDL on temporary tables won't affect transactions (pingcap#24534) *: implement tidb_bounded_staleness built-in function (pingcap#24328) executor: add correctness tests for partition table with different joins (pingcap#24673) expression: fix the spelling of word arithmetical (pingcap#24713) store/copr: balance region for batch cop task (pingcap#24521) store, metrics: Add metrics for safetTS updating (pingcap#24687) sem: add tidbredact log to restricted variables (pingcap#24701) session: fix dml_batch_size doesn't load the global variable (pingcap#24710) store/tikv: retry TSO RPC (pingcap#24682) expression, planner: push cast down to control function with enum type. (pingcap#24542) executor: add correctness tests about IndexMerge (pingcap#24674) variable: change default for DefDMLBatchSize tidbOptInt64 call (pingcap#24697) planner: add partitioning pruning tests for range partitioning (pingcap#24554) *: add option for enum push down (pingcap#24685) txn: break dependency from store/tikv to tidb/kv cause by TransactionOption (pingcap#24656) executor: enhancement for ListInDisk(support writing after reading) (pingcap#24379) kv: move TxnScope into kv (pingcap#24715) execution: fix the incorrect use of cached plan for point get (pingcap#24749) executor: add correctness tests about direct reading with indexJoin (pingcap#24497) variable: fix sysvar datarace with deep copy (pingcap#24732) *: Implementing RENAME USER (pingcap#24413) infoschema, executor: Add the deadlock table (pingcap#24524) docs: Some proposal for renaming and configurations for Lock View (pingcap#24718) planner: add range partition boundaries tests with BETWEEN expression (pingcap#24598) oracle: simplify timestamp utilities (pingcap#24688) executor: fix wrong enum key in point get (pingcap#24618) ranger: fix incorrect enum range for xxx_ci collation (pingcap#24661) executor: add some test cases about dynamic-mode and apply operator (pingcap#24683) store/tikv: remove Variables.Hook (pingcap#24758) ddl: speed up the execution time of `TestBackwardCompatibility`. (pingcap#24704) *: prepare errors for CTE (pingcap#24763) expression: support cast real/int as real (pingcap#24670) executor: add table name in log (pingcap#24666) expression: add builtin function ``json_pretty`` (pingcap#24675) ddl: make `TestDropLastVisibleColumns` stable (pingcap#24790) * ddl: make `TestDropLastVisibleColumns` stable *: support AS OF TIMESTAMP read-only begin statement (pingcap#24740) executor: Fix unstable TestTiDBLastTxnInfoCommitMode (pingcap#24779) planner: add tests for partition range boundaries for LT/GT (pingcap#24574) test: record random seed in TestIssue20658 (pingcap#24782) store/tikv/retry: define Config instead of BackoffType (pingcap#24692) config: ignore tiflash when show config (pingcap#24770) privileges: improve dynamic privs registration and tests (pingcap#24773) README: remove the link of TiDB Monthly Update (pingcap#24791) region_cache: filter peers on tombstone or dropped stores (pingcap#24726) util/stmtsummary: remove import package tikv (pingcap#24776) ddl: grammar check for create unsupported temporary table (pingcap#24723) *: update go.etcd.io/bbolt (pingcap#24799) ddl: speed up the execution time of `ddl test` and `Test Chunk pingcap#7 ddl-other` (pingcap#24780) executor: remove the unnecessary use of fmt.Sprintf (pingcap#24815) executor: fix index join panic on prefix index on some cases (pingcap#24568)

ti-chi-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 26, 2021

github-actions bot added the sig/sql-infra SIG: SQL Infra label Apr 26, 2021

xhebox force-pushed the schema_2 branch from 37c696c to b94388c Compare April 26, 2021 07:43

ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 26, 2021

xhebox force-pushed the schema_2 branch from 941b9c2 to 3b767d7 Compare April 26, 2021 10:19

xhebox requested a review from a team as a code owner April 26, 2021 11:06

xhebox requested review from lzmhhh123 and removed request for a team April 26, 2021 11:06

ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 26, 2021

github-actions bot added the sig/execution SIG execution label Apr 26, 2021

xiongjiwei reviewed Apr 27, 2021

View reviewed changes

domain/domain.go Outdated Show resolved Hide resolved

djshow832 reviewed Apr 27, 2021

View reviewed changes

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2021

xhebox force-pushed the schema_2 branch from 6abf673 to c520434 Compare April 28, 2021 03:17

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 28, 2021

djshow832 reviewed Apr 28, 2021

View reviewed changes

ddl/options.go Outdated Show resolved Hide resolved

domain/domain.go Show resolved Hide resolved

domain/domain.go Show resolved Hide resolved

domain/domain.go Show resolved Hide resolved

domain/domain.go Outdated Show resolved Hide resolved

djshow832 requested review from zimulala, AilinKid and bb7133 April 29, 2021 10:52

djshow832 reviewed May 7, 2021

View reviewed changes

infoschema/cache.go Show resolved Hide resolved

JmPotato reviewed May 10, 2021

View reviewed changes

ti-chi-bot added status/LGT1 Indicates that a PR has LGTM 1. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels May 10, 2021

xhebox and others added 3 commits May 13, 2021 14:46

*: compatibility with staleread

b1e74ef

Signed-off-by: xhe <[email protected]>

*: remove old compatibility check, and fix tests

b8d54ad

*: typo

c57ea97

Co-authored-by: xiongjiwei <[email protected]>

*: fix fmt

2a682eb

Signed-off-by: xhe <[email protected]>

Yisaer reviewed May 13, 2021

View reviewed changes

xhebox added 3 commits May 13, 2021 17:51

*: accurate test

2f7ae2f

Signed-off-by: xhe <[email protected]>

*: mistakenly deleted comment

e00f63e

Signed-off-by: xhe <[email protected]>

*: add metrics

287038d

Signed-off-by: xhe <[email protected]>

Yisaer approved these changes May 17, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 17, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 17, 2021

Merge branch 'master' into schema_2

bdf2fb4

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label May 17, 2021

*: fix panic

28c6e2b

Signed-off-by: xhe <[email protected]>

*: fix panic2

3fa0aed

Signed-off-by: xhe <[email protected]>

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 17, 2021

Merge branch 'master' into schema_2

66e4624

ti-chi-bot merged commit 2ca98e3 into pingcap:master May 17, 2021

nolouch mentioned this pull request May 19, 2021

planner, executor: supports select statement with AS OF #24613

Merged

xhebox deleted the schema_2 branch May 31, 2021 04:07

This was referenced Jun 9, 2021

Reduce the kv_get requests in stale read #25303

Closed

infoschema: maintain the snapshotTS in the infocache #25322

Merged

ti-srebot mentioned this pull request Jun 11, 2021

infoschema: maintain the snapshotTS in the infocache (#25322) #25376

Merged

xhebox mentioned this pull request Oct 29, 2021

Make stale read and history read compatible with DDL #22427

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: compatibility with staleread #24285

*: compatibility with staleread #24285

xhebox commented Apr 26, 2021 •

edited

Loading

xhebox commented Apr 26, 2021

xhebox commented Apr 26, 2021

djshow832 left a comment

xhebox commented May 7, 2021

JmPotato May 10, 2021

xhebox May 10, 2021

xhebox May 13, 2021

JmPotato May 10, 2021

xhebox May 10, 2021 •

edited

Loading

JmPotato May 10, 2021 •

edited

Loading

xhebox May 10, 2021

djshow832 commented May 10, 2021

Yisaer left a comment

Yisaer May 13, 2021

xhebox May 13, 2021

Yisaer left a comment

Yisaer May 17, 2021

xhebox May 17, 2021 •

edited

Loading

ti-chi-bot commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

ti-chi-bot commented May 17, 2021

ti-chi-bot commented May 17, 2021

*: compatibility with staleread #24285

*: compatibility with staleread #24285

Conversation

xhebox commented Apr 26, 2021 • edited Loading

What problem does this PR solve?

Check List

Release note

xhebox commented Apr 26, 2021

xhebox commented Apr 26, 2021

djshow832 left a comment

Choose a reason for hiding this comment

xhebox commented May 7, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xhebox May 10, 2021 • edited Loading

Choose a reason for hiding this comment

JmPotato May 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djshow832 commented May 10, 2021

Yisaer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yisaer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xhebox May 17, 2021 • edited Loading

Choose a reason for hiding this comment

ti-chi-bot commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

xhebox commented May 17, 2021

ti-chi-bot commented May 17, 2021

ti-chi-bot commented May 17, 2021

xhebox commented Apr 26, 2021 •

edited

Loading

xhebox May 10, 2021 •

edited

Loading

JmPotato May 10, 2021 •

edited

Loading

xhebox May 17, 2021 •

edited

Loading